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Innovations  in  Software  Engineering  Education:  An  Experimental  Study  of 
Integrating  Active  Learning  and  Design-based  Learning 


Abstract 

Significant  advancements  have  been  made  in  engineering  education  in  recent  years.  An  important 
outcome  of  these  advancements  is  the  integration  and  extension  of  fundamental  pedagogies  as  part  of 
engineering  curricula,  as  well  as  the  need  for  continued  research  into  the  effectiveness  of  these 
pedagogies  on  students’  learning  within  engineering  know  ledge  domains.  In  this  paper,  we  focus  on  an 
engineering  educational  research  study  in  the  domain  of  software  engineering.  This  study  considers  the 
important  research  question  of  the  efficacy  of  traditional  lecture-homework-project  teaching  approaches 
compared  to  peer-to-peer  active  learning  when  combined  with  design-based  learning  approaches.  The 
focus  of  the  study  is  on  participants’  qualitative  assessment  and  self-reported  motivation  for  learning 
fundamental  programming  and  software  engineering  principles  when  comparing  the  teaching  approaches. 
The  participants  are  divided  into  a  control  group  focusing  on  a  traditional  teaching  approach  and  an 
experimental  group  where  the  teaching  includes  a  unique  integration  of  peer-to-peer  learning  and  design- 
based  pedagogy.  Results  from  this  study  demonstrate  that  software  engineering  concepts  taught  in  either 
pedagogical  framework  are  well  received  by  the  students,  and  both  groups  show  distinct  improved 
performance  relative  to  entering  the  course.  However,  a  peer-to-peer  active  environment  and  design- 
based  learning  framework  are  received  with  much  greater  interest,  engagement,  sense  of  relevance,  and 
intrinsic  motivation,  especially  related  to  particular  classroom  and  lab  experiences. 


I-  Introduction 

Over  the  last  two  decades*  significant  advancements  have  been  made  in  the  development  of 
active  learning  approaches  in  many  fields  of  engineering.  These  efforts  focus  on  transforming 
course  content  from  passive*  traditional  classroom  environments  to  modes  where  the  students 
take  a  much  more  dynamic  and  participatory  role.  Similar  advancements  have  been  made  in 
project-based  learning  (PBL)  and  design-based  learning  (DBL),  In  these  cases,  students  leam  in 
an  application-based  environment*  typically  in  teams*  and,  in  the  preferred  mode,  directed  by 
open-ended  problems  where  no  single  right  answer  exists. 

While  the  literature  reports  a  number  of  examples  of  active  learning  and  design-based  learning 
approaches  within  engineering,  few  investigations  are  reported  regarding  their  integration.  This 
void  especially  exists  in  certain  disciplines  of  engineering,  such  as  software  design  -  typically 
found  in  computer  engineering*  electrical  engineering,  and  computer  science  programs. 

Students  in  these  programs,  and  in  particular  Computer  Science  or  Software  Engineering 
courses,  are  often  excited  at  first,  but  gradually  lose  interest  and  motivation  while  tackling  more 
and  more  difficult  theoretical  problems.  As  a  result,  it  is  extremely  important  to  explore  methods 
to  consistently  engage  students  while  equipping  them  with  fundamentals,  technical  skills, 
innovation  skills,  and  learning  skills.  Research  shows  that  design-based  engineering  curriculums 
are  able  to  intrigue  students7  curiosity  in  exploring  a  new  area;  while  an  active  learning  approach 
enables  consistent  engagement  to  students  through  probing  discussions  and  activities.  Hence, 
integrating  both  design-based  learning  and  active  learning  into  Software  Engineering  curricula 
could  potentially  deliver  an  enriching  and  engaging  learning  experience  to  students  where  they 
exhibit  increased  motivation*  perceive  greater  relevance  of  course  content,  and  demonstrate  high 
interest  in  subject  matter. 


In  this  paper,  we  consider  a  systematic  investigation  of  combining  active  learning  and  design- 
based  learning  for  the  instruction  of  software  engineering  content.  The  investigation  entails  an 
experiment  based  on  two  scenarios:  a  control  group,  focusing  on  the  traditional  approaches,  and 
an  experimental  group,  focusing  on  active  learning  through  peer  to  peer  instruction  and  the 
integration  of  design  project  modules.  For  the  purposes  of  this  study  and  educational  needs  of 
the  students,  the  experimental  environment  is  chosen  to  be  an  intermediate-level  short  course  on 
objective-oriented  programming  in  Java.  The  control  group  follows  a  traditional  learning 
approach,  where  content  is  taught  in  a  combination  of  lectures,  labs,  and  homework  assignments. 
The  experimental  group,  on  the  other  hand,  learns  content  through  pre-assigned  readings,  peer- 
peer  active  presentations  and  discussions  of  course  content,  faculty-led  follow-up  discussions  of 
content  at  a  peer  level,  connections  of  provocative  and  real-life  examples  to  motivate  course 
content  (i.e.,  “show  and  tell”),  and  design-based  problems  integrated  throughout  classroom  and 
out-of-class  activities. 

Therefore,  present  research  intends  to  identify  if  there  is  significant  difference  between  two 
learning  approaches  through  following  metrics:  1)  participant’s  pre  and  post-test  performance, 
and  2)  learning  approach  feedback  about  engagement,  student  interest,  and  motivation  for 
software  engineering  content. 


II.  Related  Research  and  Pedagogical  Underpinnings 

Engineering  education  continues  to  change  as  we  encounter  more  interdisciplinary  learning  and 
create  access  for  a  wider  variety  of  students  to  be  strive  for  success  within  higher-education 
classrooms  (Adams,  et  ah,  2004;  Anderson  and  Northwood,  2002).  In  better  understanding  the 
intricacies  of  students  ’  learning,  educators  can  improve  the  teaching  of  engineering  concepts.  A 
variety  of  salient  educational  theories  guide  the  creation,  assessment,  and  improvement  of 
engineering  domain  content,  within  an  active-learning  context.  These  learning  theories  range 
from  how  information  is  presented  and  processed  to  understanding  an  individual  student’s 
personality  and  preferences.  Here  we  focus  on  developments  in  active  learning  approaches 
particularly  tailored  and  found  prominently  in  engineering  education  curricula  and  /  or  research. 

Ill  Active  Learning,  Interactive  Engagement,  &  Constructivist  Theories 

Active  learning  approaches  improve  students’  overall  learning,  a  view  shared  generally  by 
faculty  teaching  engineering  education  (Aglan,  1996).  There  is  considerable  literature  that 
addresses  the  advantages  of  using  active  learning  in  STEM  curriculum  (Aglan,  1996;  Bonwell; 
Dennis,  2001;  Eder,  2001;  Hsi,  1995;  Holzer,  2000;  Linsey,  2006,  2007,  2009;  Mayer,  2002; 
Meyer,  1994;  Prince,  2004;  Stice,  1987;  Talley,  2007;  Welsh,  2007;  Wood,  2000,  2001,  2002, 
2004;  Barr,  2000;  Bean,  2001).  These  literature  sources  show  that  students’  motivation  and 
learning  are  simultaneously  enhanced  by  the  incorporation  of  active  learning  into  the  classroom. 

The  foundation  of  active  learning  is  a  constructivist  teaching  philosophy  and,  in  particular,  social 
constructivism.  Through  the  interaction  with  ideas,  concepts,  materials,  and  other  artifacts, 
students  construct  their  knowledge.  This  approach  seeks  to  alter  the  mode  of  knowledge  and 
conceptual  understanding  through  student  construction  as  opposed  to  passive  reception. 
Modules  may  be  framed  within  a  constructivist  framework  in  an  effort  to  create  engineering 
education  experiences  that  (1)  help  students  to  construct  deeper  understanding  of  theoretical 
concepts  in  connection  to  practical  experiences;  (2)  facilitate  students’  engineering  skills;  and  (3) 


develop  students’  capabilities  and  dispositions  for  engaging  in  collaborative  project-based 
inquiry  and  critical  thinking.  To  assimilate  new  information  and  incorporate  it  into  the  existing 
knowledge,  students  need  to  restructure  their  knowledge  for  themselves,  which  can  be 
accomplished  through  active  learning.  A  number  of  tenants  underlie  this  teaching  philosophy, 
including  (Knight,  2004):  students  take  direct  responsibility  for  their  knowledge,  proactively 
engaging  in  the  study  of  their  texts  and  reference  materials,  participation  and  leadership  in  course 
activities,  completing  assignments,  laboratories,  and  exploration  in  the  field;  the  instructor 
assumes  more  of  a  role  of  a  facilitator:  “a  guide  on  the  side,  not  a  sage  on  the  stage;  students 
receive  immediate  feedback  and  guidance  on  their  work  and  class  contributions;  and  students 
invest  and  spend  a  significant  portion  of  class  time  engaged  in  concrete  experiences  (doing)  and 
reflections,  contemplation  and  critically  thinking,  and  discussing  the  topic  area  of  the  course. 
This  approach  is  opposed  to  a  passive  listening  of  the  topic  from  others. 

Active  learning  or  interactive  engagement  does  not  comprise  a  single  approach  but  many 
approaches  may  be  executed  through  a  variety  of  modes  and  media.  Exemplar  delivery  modes, 
with  extensive  testing  (e.g.,  in  the  area  of  STEM  physics  (Laws,  1997)),  include  Cooperative 
Groups  (Heller,  1992a,  1992b),  Socratic  Dialogue  Inducing  Labs  (Hake,  1987,  1992),  Interactive 
Demonstrations  (Sokoloff,  1997),  Peer  Instruction  (Mazur,  1997),  Think/Pair/Share  (Van 
Heuvelen,  1999),  Tutorials  (McDermott,  1994,  1998),  and  Hands-on  Activities  (Linsey,  2006 
2007,  2009). 

Extensive  empirical  studies  have  been  carried  out  to  understand  the  role  of  active  learning.  Clear 
evidence  exists  that  the  lecture  mode  of  instruction  and  passive  reception  by  students  is  not 
effective  at  leading  to  student  understanding.  Many  contributory  factors  explain  this  result 
(Knight,  2004):  Students  are  not  good  listeners  in  a  passive  instructional  setting.  They  simply  do 
not  know  how  to  listen.  Critical  thinking  skills  cannot  be  developed  or  applied  to  subject  matter 
due  to  the  pace  of  content  presentation.  The  meaningful  attention  span  of  individuals  is  10-15 
minutes.  Many  orally  delivered  lectures  simply  reiterate  material  in  the  text  or  references  for  a 
course  or  topic  area.  Lectures  typically  focus  on  abstractions  not  concrete  experiences  with  a 
subject  matter. 

In  the  area  of  STEM  physics,  extensive  testing  between  conventional  teaching  approaches  versus 
active  learning  modes  showed  more  than  a  two  times  (100%)  improvement  regarding  the 
conceptual  understanding  of  students  with  a  Newtonian  Force  Concept  Inventory.  These  results 
are  striking  and  correlated  closely  with  students5  abilities  to  develop  and  employ  good  problem 
solving  skills  (Hake,  1998).  Depending  on  one’s  teaching  styles,  the  intent  is  to  integrate 
interactive  engagement  throughout  innovative  initiatives  in  courses  and  the  cultural  setting  of  the 
students’  educational  environment. 

II.  2  Integrated  Learning  Experiences:  the  Kolb  Cycle 

The  Kolb  model  shown  in  Fig.  1  describes  an  entire  cycle  around  which  a  learning  experience 
progresses.  The  goal  is  to  structure  learning  activities  that  will  envelope  the  entire  cycle, 
providing  the  maximum  opportunity  for  foil  comprehension  and  self-learning.  Increasing 
engineering  educators  are  wielding  this  model  to  evaluate  and  enhance  engineering  teaching. 
Software  engineering  models  may  be  designed  to  provide  learning  experiences  motivated  by  the 
Kolb  cycle  that  are  not  well  met  with  traditional  course  instruction.  Specifically,  software 
engineering  learning  modules  may  be  developed  with  need-based,  service  learning,  or  industry  or 
research  relevant  problems.  This  provides  the  “Concrete  Experience”  component  of  the  cycle  in 


a  similar  maimer  as  a  case  study.  The  “Reflective  Observation”  part  of  the  cycle  may  be 
accomplished  by  providing  key  times  for  student  questions,  critique,  and  assessment  questions 
throughout  the  learning  modules.  These  observational  opportunities  may  be  designed  to 
encourage  the  students  to  reflect  on  the  innovation  history,  processes,  problem,  theoretical 
frameworks,  ideas,  and  /  or  decisions.  The  “Abstract  Hypothesis  and  Conceptualization” 
component  of  the  Kolb  cycle  may  be  addressed  through  the  use  of  the  course  content  itself,  in 
addition  to  supporting  conceptual  course  material.  A  concept  inventory  may  lead  the 
construction  and  assessment  goals  of  this  Kolb  component,  led  by  student  peer-to-peer 
interactions.  Finally,  the  “Active  Experimentation”  part  of  the  cycle  may  be  met  when  the 
students  will  be  asked  to  interact  and  iteratively  solve  distinct  modules,  either  reproducing 
known  concepts  in  the  subject  matter,  or  addressing  a  building  block  of  course  concepts. 

II.  3  Design-Based  Learning  (DBL) 

Significant  advancements  have  been  made  in  design-based  learning  (DBL),  especially  in  the  context  of 
active-  and  hands-on  learning  (Sheppard,  1992;  Dym,  1994,  1999,  2003-2005;  Crawford,  1994;  Eder, 
1994;  Daily,  1993;  His,  1995;  Dutson,  1997;  Otto,  1998,  2001;  Barr,  2000;  Tavakoli,  2000;  Newman, 
2001;  Dennis,  2001;  Jensen,  2000,  2005;  Bilen,  2002;  Blessing,  2002;  Campbell,  2002;  Ulrich,  2004; 
Green,  2004;  Charyton,  2009;  Linsey,  2010;  Markman,  2011;  White,  2011;  Wood,  2000,  2001,  2002, 
2005,  2012).  These  advancements,  in  various  direct  and  indirect  ways,  provide  frameworks  for  learning 
through  open-ended  problems,  creative  problem-solving,  and  engagement  in  service-learning  and  society- 
based  projects.  The  work  reported  in  this  paper  builds  upon  these  foundations  and  advancements. 
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Figure  1 .  Kolb  cycle  for  integrated  learning  experiences  in  assimilating  and  processing  information 


11.4  Performance  assessment  design 


Performance  assessments  are  used  to  examine  the  quality  of  studies  (Messick,  1994).  Depending  on  the 
nature  of  the  studies,  performance  assessment  can  be  product-driven  or  performance-driven.  Product- 
driven  assessment  evaluates  the  outcome  of  the  domain  whereas  performance-driven  assessment  focuses 
on  the  quality  of  performance  in  the  process.  In  cases  where  evaluation  of  the  product  or  performance  is 
the  focus,  replicability  and  generalizability  are  negligible.  In  the  performance  assessment  of  competencies, 
performance-driven  assessment  design  has  to  consider  generalizability  and  replicability  in  order  to 
measure  how  consistent  the  meaning  of  the  issue  is  likely  to  be.  In  this  work,  we  build  on  Messick’s 
contributions  to  develop  assessment  instruments. 

III.  Experimental  Study  Design 

Building  on  the  related  research  and  pedagogical  underpinnings  in  Section  II,  we  consider  here 
the  design  of  the  experimental  study.  The  primary  hypothesis  of  the  research  study  is  as  follows: 
“ There  exists  significant  improvement  in  the  engagement,  student  interest,  and  motivation  for 
software  engineering  content  using  an  integrated  approach  of  active  and  deign-based  learning 
compared  to  traditional  teaching  approaches.  ”  Traditional  approaches  refer  to  a  combination 
of  lectures,  tutorials  and  lab  sessions  for  a  software  engineering  course. 

To  test  this  hypothesis,  Hie  experimental  study  included  the  design  of  software-engineering 
course  content,  coordination  of  the  study’s  control  (traditional)  and  experimental  (active-DBL) 
groups,  and  development  of  the  assessment  and  evaluation  instruments. 

III.  I  Course  Design 

Content  Framework.  For  the  purposes  of  this  study,  a  five-day  short  course,  offered  during 
independent  activity  periods  (IAPs)  as  vacation  period  where  the  students  do  not  take  curriculum 
courses,  is  chosen  as  the  venue  to  carry  out  the  experimental  study.  This  venue  provides  a 
number  of  advantages  to  carry  out  the  study,  namely:  (i)  new  courses  may  be  offered  with 
relative  ease  compared  to  core  curriculum  courses  or  electives;  (ii)  greater  experimental  control 
may  be  obtained  relative  to  consistency  in  instructor  talents  and  skill  set,  environmental 
conditions  across  the  experiment,  and  the  availability  of  student  time  and  resources;  (3) 
participants  in  the  study  are  volunteers,  with  personal  choice  and  interests  as  the  motivation  to 
enroll  in  the  course;  and  (4)  significant  contact  hours,  in  this  case  approximately  seven  hours  per 
day  for  three  to  five  days,  exist  with  the  students  in  a  relatively  short  overall  timeframe. 

The  software  engineering  topic  chosen  tor  the  five-day  short  course  is  object-oriented 
programming  (OOP),  where  the  focus  language  is  Java.  The  learning  objectives  for  the  students 
focus  on  gaining  intermediate  mastery  of  OOP  concepts  and  apply  these  concepts  in  a  designette 
project  (Wood,  2012).  The  course  concepts  are  explored  during  the  first  three  days  of  the  short 
course,  followed  by  two  days  of  designette  project  work  and  presentation  of  the  results.  In  view 
of  the  course  timeframe,  a  balance  is  sought  between  breadth  and  depth  of  the  course  content 
while  designing  the  course  framework.  While  obtaining  programming  expertise  is  one  of  the 
learning  objectives  of  the  course,  we  expect  the  students  to  be  familiar  with  some  basic 
knowledge  of  Java  programming,  such  as  its  syntax  and  the  use  of  Integrated  Development 
Environment  (IDE).  This  expectation  is  based  on  the  type  of  students  typically  interested  in 
such  IAP  software  engineering  experiences  and  validated  by  the  volunteer  students  that  enrolled 
in  the  course.  In  spite  of  this  expectation,  instructors  still  devoted  a  minimum  amount  time  in 


reviewing  basic  background  knowledge.  This  review  served  as  a  bridging  session  from  the  basics 
to  intermediate  level  (and  even  some  advanced)  OOP  concepts. 

The  main  course  framework  consists  of  three  sections.  They  are  Fundamental  Object-Oriented 
Concepts,  Java  Libraries,  and  Integrated  Techniques.  The  Object-Oriented  Concepts  section 
seeks  the  study  of  Objects,  Classes,  Encapsulation,  Inheritance  and  Polymorphism.  These 
concepts  are  crucial  for  students  to  design  and  engineer  interactive,  effective  and  reusable  codes 
to  build  sustainable  system  architecture. 

Java  Libraries  refers  to  pragmatic  Java’s  packages  and  functions  that  are  pre-developed  and 
integrated  with  the  Java  Development  Kit  (JDK).  In  the  designed  course,  we  introduce  the  basic 
packages  and  functions  such  as  the  Collection  frameworks  and  Java  I/O  (Input/Output). 
Although  these  libraries  belong  to  the  Java  language,  other  object-oriented  languages,  such  as 
Microsoft’s  C#,  too  have  similar  libraries.  As  a  result,  familiarizing  with  the  built-in  classes  and 
interfaces  is  very  helpful  for  software  engineers  to  tackle  problems  while  building  on  well- 
known  modules. 

Last  but  not  least,  the  Integrated  Techniques  section  introduces  the  use  of  Generics  and 
Enumerations.  These  topics  strengthen  the  flexibility  developing  software.  By  acquiring  skills 
with  these  two  techniques,  students  are  able  to  better  tackles  uncertainties,  such  as  receiving 
ambiguous  object  types  in  various  situations. 

Figure  2  shows  the  organization  and  breakdown  of  this  course  content  framework.  This 
framework,  while  specifically  designed  for  the  experimental  study  and  short  course,  relates  and 
overlaps  significantly  with  semester-level  courses  on  object-oriented  programming. 

Concept  Inventory.  In  creating  the  course  content  framework  as  shown  in  Fig.  2,  we  developed 
a  concept  inventory  for  the  focus  area  of  object-oriented  programming.  Concept  inventories 
have  been  developed  in  a  number  of  fields,  such  as  mechanics,  geosciences,  physics,  signals  and 
systems,  astronomy,  statistics,  computer  engineering  (CECI,  2012),  and  chemistry.  To  the  best 
of  our  knowledge,  a  software  or  computer  engineering  concept  inventory  has  not  been  developed 
for  object-oriented  programming. 

We  systematically  developed  an  object-oriented  programming  concept  inventory  to  support  the 
course  and  experimental  study  design.  The  concept  inventory  was  formed  based  on  OOP 
literature  and  text  books  in  the  field.  In  particular,  it  was  developed  with  reference  to  six 
textbooks  we  used  in  concert  with  the  course  development.  These  six  textbooks  are  (a)  Thinking 
in  Java  Fourth  Edition  (Eckel,  2003),  (2)  Head-First  Java  Second  Edition  (Sierra,  2005),  (3)  The 
Object-Oriented  Thought  Process  Third  Edition  (Weisfeld,  2009),  (4)  Java  -  How  To  Program 
Ninth  Edition  (Deitel,  2012),  (5)  Effective  Java  Second  Edition  (Bloch,  2008),  and  (6)  The  Sun- 
Certified  Java  Programmer  6.0  (SCJP  6.0  or  310-065)  Exam  Study  Guide  (Maiming,  2009). 
Other  supporting  references  include  (Winblad,  1990;  Budd,  1991;  Martin,  1992). 

The  concept  inventory  includes  key  object-oriented  theories,  principles,  and  practical  problems. 
It  includes  both  theory  and  application  approaches  on  each  topic.  It  then  formed  the  basis  of 
designing  the  course  content  framework,  as  shown  in  Fig.  1,  in  addition  to  pre-  and  post-test 
assessment  and  evaluation  instruments  for  the  course  and  study.  Though  we  developed  the 
concept  inventory  systematically  as  the  intersection  of  noted  key  concepts  and  student 
conceptual  misunderstandings,  the  concept  inventory,  shown  by  Fig.  1  and  our  pre-  and  post-test, 
may  be  evolved  and  improved  significantly.  One  needed  evolution  is  to  solicit  experts  across 


software  engineering,  and  object-oriented  programming  and  design  in  particular,  to 
collaboratively  develop  a  meaningful  and  long-lasting  inventory.  We  could  collect  questions 
through  surveys  and  long-distance  collaborations,  gather  relevant  comments  and  suggestions, 
and  then  systematically  verify  and  validate  the  resulting  concept  inventory.  The  concept 
inventory  developed  in  this  work,  as  exemplified  by  Fig.l  and  our  pre-  and  post-test,  represents  a 
significant  step  toward  a  robust  concept  inventory  for  OOP. 
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Figure  2.  Object  Oriented  Programming  Course  Content  Framework  for  Experiment  Study 
III.2  Coordination:  Course  and  Experimental  Study 

Participants.  The  short  course  was  offered  to  students  from  a  variety  of  educational  and 
experiential  backgrounds.  Recruitment  included  senior  students  from  Singapore  Polytechnic,  a 
secondary-level  engineering  diploma  granting  institution,  and  freshmen  from  Singapore 
University  of  Technology  and  Design  (SUTD).  Figure  3  lists  relevant  demographic  data 
regarding  the  participants.  The  background  of  the  students  from  these  institutions  was  viewed  to 
be  commensurate  because  of  the  engineering  diploma  emphasis  of  Singapore  Polytechnic.  Of 


course,  some  age  differences  exist  in  the  students,  which  we  attempted  to  control  through  the 
distribution  of  students  amongst  control  and  experimental  groups. 

As  the  short  course  was  designed  to  be  intermediate  level,  participation  of  the  course  required 
students  to  have  basic  object-oriented  programming  background  or  experience  in  other 
programming  practices.  While  registering  for  the  course,  students  completed  a  pre-survey  to 
indicate  their  programming  background.  These  data  were  used,  in  part,  as  an  indicator  to 
distribute  participants  across  a  control  group  and  experimental  group  for  the  experimental  study. 

All  students  from  Singapore  Polytechnic  completed  at  least  one  object  oriented  programming 
module  as  part  of  their  curriculum.  Some  have  experience  in  other  programming  modules  such 
as  Java  programming,  data  structures,  and  algorithm  and  systems  development. 

All  students  from  SUTD  are  first  freshmen.  AH  except  one  student  had  object  oriented 
programming  experience  in  either  Java  or  C++ .  The  remaining  student  had  limited  programming 
experience. 
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Figure  3.  Participants’  background  and  relevant  demographics 


Students  that  volunteered  for  enrollment  are  brought  together  for  a  course  orientation.  The 
students  are  told  that  the  short  course  would  include  a  pedagogical  experiment,  based  on 
voluntary  participation.  However,  information  such  as  how  both  classes  would  be  conducted  and 
their  curricuhims  were  not  disclosed  to  students. 

At  the  end  of  the  orientation,  all  students  are  given  a  pre-test  based  on  the  concept  inventory 
developed  as  part  of  the  project.  Upon  completion,  the  students  are  then  assigned  to  either  the 
control  or  experimental  groups,  where  they  are  evenly  distributed  according  to  educational  level 
and  skills  in  software  programming  based  on  the  data  in  the  pre-survey. 

After  separating  the  participants  into  control  and  experimental  groups,  the  short  course  was 
implemented  over  a  one-week  period.  The  participant  groups  remained  segregated  for  course 
content  instruction  and  course  exercises  until  the  final  day  of  the  course  where  the  students 
presented  results  of  a  final  designette  project  (Wood,  2012).  Faculty  members  teaching  the  short 
course  were  recruited  to  have  similar  expertise,  enthusiasm,  and  content  knowledge,  independent 


of  the  control  and  experimental  group  structure.  They  instructors  were  given  the  same  content 
material,  but  were  not  informed  about  the  pedagogical  structure  being  implemented  in  the  other 
instructor’s  group. 

Control  Group.  The  control  group  uses  a  traditional  approach  in  teaching.  It  includes  lectures, 
lab  sessions  and  homework.  On  the  other  hand,  the  experimental  group  uses  active  learning 
through  peer-to-peer  instruction  and  the  integration  of  design  project  modules.  Figure  4  shows 
the  mapping  of  the  course  content  framework,  Fig.l,  to  both  the  control  group  and  experimental 
group  pedagogical  structures. 

The  structure  for  the  control  group  includes  a  combination  of  lecture,  lab  and  homework,  carried 
out  in  five  consecutive  days.  In  the  morning  session  of  each  day,  the  students  receive  two  3-hour 
lectures  with  a  15 -minute  break  in  between.  The  instructor  gives  the  lectures  using  whiteboards 
and  markers. 


Course  Implementation  Framework 

Control  Group 

Experimental  Group 

Duration 

Content 

Time  (min) 

Item 

Duration 

Content 

Time  (min) 

1 

9:00-10:30 

Lecture 

90 

1 

9:00-10:00 

Peer-to-peer 
interactive  Discussion 

60 

2 

10:3010:45 

Tea  Break 

15 

2 

10:00-10:15 

Tea  Break 

15 

Day  1 

3 

10:45-12:15 

lecture 

90 

3 

10:15-12:00 

Peer-to-peer 
Interactive  Discussion 

45 

4 

12:15-13:00 

Lunch  Break 

45 

4 

12:00-12:45 

Lunch  Break 

45 

5 

13:00-16:00 

Lab 

180 

5 

12:4546:00 

Coiaborative  tab 

195 

1 

9:00-10:30 

Lecture 

90 

1 

9:00-10:00 

Peer-to-peer 
Interactive  Discussion 

60 

2 

10:30-10:45 

Tea  Break 

15 

2 

10:00-10:15 

Tea  Break 

15 

Day  2 

3 

10:45-12:15 

Lecture 

90 

3 

10:1542:00 

Peer-to-peer 
Interactive  Discussion 

45 

MM 

12:15-13:00 

Lunch  Break 

45 

4 

12:00-12:45 

Lunch  Break 

45 

n 

13:00-16:00 

lab 

180 

5 

12:4546:00 

Coiaborative  Lab 

195 

i 

9:00-10:30 

Lecture 

90 

D 

9:00-10:00 

Peer-to-peer 
interactive  Discussion 

60 

2 

10:30-10:45 

Tea  Break 

IS 

2 

10:00-10:15 

Tea  Break 

15 

DayB 

3 

10:45-12:15 

lecture 

9G 

3 

10:15-12:00 

Peer-to-peer 
Interactive  Discussion 

45 

4 

12:15-13:00 

Lunch  Break 

45 

4 

12:00-12:45 

Lunch  Break 

45 

5 

13:00-16:00 

Designette 

180 

S 

12:45-13:30 

Coiaborative  lab 

45 

6 

12:4346:00 

Designette 

150 

Day  4 

MM 

9.00-16:00 

Design  ette 

540 

1 

9:00-16:00 

Designette 

540 

D 

10:00-12:00 

Presentation 

(ISmin/team) 

120 

1 

10:00-12:00 

Presentation 
(15m  in/team} 

120 

Day  5 

2 

12:00-12:45 

Lunch 

45 

2 

12:00-12:45 

Lunch 

45 

3 

12:45-13:30 

Post-test 

45 

3 

12:4543:30 

Post-test 

45 

4 

13:30-13:45 

Survey 

15 

4 

13:30-13:45 

Survey 

15 

5 

13:45-14:00 

Debrief 

20 

5 

13:4544:00 

Debrief 

20 

Figure  4.  Mapping  of  course  content  framework  (Fig.  1 }  to  course  pedagogy  (Course  Implementation 

Framework) 


In  the  afternoon  session,  the  students  work  in  the  lab,  where  they  are  given  a  problem  set 
selected  from  “Thinking  in  Java”  (Eckel,  2003),  and  a  reading  assignment  that  is  relevant  to  the 
topics  of  the  next  day.  A  post-doctoral  research  associate  works  in  the  lab  to  assist  the  students 
and  answer  questions.  The  students  are  expected  to  complete  half  to  two-thirds  of  the  problems 


within  the  3-hour  tab  session,  leaving  the  remaining  work  and  reading  assignment  to  be 
completed  as  homework. 

The  contents  of  the  lectures  are  organized  as  follows.  A  general  overview  of  programming  is 
given  in  the  beginning,  which  includes  an  introduction  of  the  purpose  and  essence  computer 
programming,  and  a  brief  history  of  programming  languages.  Java  is  identified  in  a  taxonomy  of 
programming  languages.  A  key  concept  introduced  in  this  lecture  is  that  programming  is  a  means 
by  which  humans  control  computers,  and  hence  the  human  model  and  the  computer  model  are 
two  sides  of  the  story  about  computer  programming. 

For  object-oriented  programming,  and  in  particular  Java,  the  human  model  is  object  oriented 
programming  (OOP)  and  the  computer  model  is  the  Java  virtue  machine  on  an  abstract  computer 
architecture.  The  remainder  of  the  lecture  series  is  given  by  telling  both  sides  of  the  story,  and 
how  Java  syntax  and  concepts  are  designed  to  connect  them. 

One  lecture  is  given  on  the  basic  concepts  of  OOP,  including  encapsulation,  inheritance  and 
polymorphism.  An  example  is  used  to  explain  the  above  concepts,  with  pseudocode  represented 
in  UML.  The  difference  between  composition  and  inheritance  is  discussed,  highlighting  the 
differences  and  possible  design  choices.  Multiple  inheritance  is  explained  after  the  introduction 
of  single  inheritance,  and  parameterized  types  are  introduced  at  the  end  of  this  lecture. 

On  the  other  side  of  the  story,  one  lecture  is  given  on  the  basic  computer  architecture  and  the 
Java  virtual  machine.  A  computer  word  is  explained  after  a  brief  review  of  binary  numbers. 
Computer  architecture  is  discussed  in  terms  of  a  CPU  and  registers,  memory,  I/O  devices  and  the 
central  bus.  A  reflection  of  the  first  lecture  is  given  by  explaining  why  this  architecture  can 
achieve  what  computers  do  today,  including  robotics,  multimedia  and  others.  Chip  instruction 
sets  were  discussed  during  the  introduction  of  a  CPU,  and  simple  instructions  of  the  Intel  X86 
chipset  were  reviewed.  The  organization  of  a  program  in  the  memory  is  discussed,  with  the  split 
of  the  data  section  into  a  stack  and  a  heap  explained  in  elaboration. 

The  remainder  of  the  lectures  consists  of  two  main  topics,  including  Java  Syntax,  00  concepts 
and  libraries.  The  syntax  of  Java  is  introduced  in  three  main  sections.  The  first  section  includes 
primitive  types  and  values,  expressions,  statements,  control  flow,  functions,  classes,  garbage 
collection,  code  compilation,  packages  and  comments.  The  basic  concepts  were  organized  in  the 
sequence  of  a  traditional  textbook,  so  that  after  all  concepts  were  introduced,  the  students  could 
write  a  complete  Java  program.  Special  classes  such  as  String  and  Array  are  introduced  when 
Classes  are  introduced.  Boxing  and  Enum  are  also  introduced  in  this  section.  Programming 
concepts  are  always  discussed  with  their  JVM  behavior. 

The  second  section  consists  of  access  control,  inheritance,  polymorphism,  interface,  inner  class 
and  generics.  These  concepts  were  introduced  in  correspondence  with  the  Java  Syntax 
introduced  previously.  For  example,  interface  and  inner  class  are  introduced  as  implementations 
of  multiple  inheritances.  On  the  machine  side,  JVM  behavior  of  each  syntax  type  is  discussed. 
The  third  section  consists  of  language  features  including  exceptions,  reflection  and  annotations. 
Again  both  the  design  of  programming  paradigms  and  the  implementation  on  the  Java  virtual 
machine  are  explained.  The  second  main  topic  is  Java  libraries,  where  containers  and  file  I/O  are 
highlighted. 

The  philosophical  basis  for  the  traditional  lecturing  includes  the  laying  out  of  a  big  picture  and 
consistency  in  storytelling.  The  lecturer  puts  all  relevant  knowledge  under  a  broad  theme,  and 


discusses  each  topic  with  regard  to  the  theme.  The  students  have  a  comparatively  uninteixupted 
reception  ot  the  whole  flow  of  concepts  from  a  single  consistent  point  of  view. 

During  our  control  group  lectures,  questions  were  encouraged,  and  the  students  were  engaged. 
The  lecturer  asked  relevant  questions  to  the  students  after  the  introduction  of  each  concept  to 
make  sure  that  most  students  understood  the  concepts.  Missing  of  particular  concepts  by  a  small 
portion  of  students  was  deemed  as  understandable  and  acceptable  by  the  instructor,  while  the 
lecturer  always  made  sure  that  the  students  knew  the  big  picture  so  that  they  would  be 
encouraged  to  learn  by  themselves  the  missing  content,  knowing  their  position  in  the  broad 
lecture  theme. 

A  potential  disadvantage  of  the  control  group  was  that  the  lectures  and  lab  sessions  were  given 
completely  separately,  without  extensive  communication  between  the  lecturer  and  the  teaching 
assistant.  For  future  experiments  we  would  make  the  lab  sessions  more  closed  linked  to  the 
lectures  by  having  the  lecturer  design  the  lab  work,  and  increasing  communication  between  the 
lecturer  and  the  RA. 

Experimental  Group.  The  course  framework  for  the  experimental  group  consists  of  four  main 
components:  pre-reading,  peer-to-peer  interactive  discussion,  collaborative  lab,  and  integrated 
design-based  learning  designettes  (Wood,  2012).  Figure  5  shows  an  experimental  group  setting 
where  students  are  paired  into  teams  of  peer-to-peer  learners,  and  students  initially  direct  the 
content  in  a  given  session. 


Figure  5.  Experimental  group  setting 


Pre-reading  Materials.  The  students  are  given  a  set  of  pre-reading  material  one  day  before  the 
actual  lessons.  Each  set  consists  of  topics  that  are  related  to  the  next  day’s  lesson.  From  pre- 
reading,  the  students  are  expected  to  develop  an  overview  of  the  topics  before  attending  the  class 
on  the  following  day.  These  pre-readings  provide  background  knowledge  to  the  students  in  order 


to  actively  participate  in  the  peer-to-peer  interactive  discussions.  The  pre-reading  materials  also 
serve  as  a  reference  library  when  students  have  questions  or  meet  obstacles  during  the 
collaborative  lab  sessions  and  integrated  designettes.  As  a  result,  the  pre-reading  material  is 
designed  to  have  simple  presentation,  comprehensive  information  and  understandable  language. 


The  sets  of  pre-reading  material  are  developed  based  on  the  concept  inventory.  We  composed 
these  topics  based  on  related  daily  interactive  discussions,  initially  led  by  student  peer-to-peer 
teams.  Figure  6  show  exemplar  pre-reading  materials  based  on  selected  course  inventory  topics, 
Figure  7  shows  exemplar  practical  exercises  included  as  part  of  the  pre-reading  materials,  and 
Figure  8  shows  sample  codes  as  part  of  the  pre-reading  materials  that  express  OOP  concepts. 


A  class  is  not  an  object. 

(but  tt'*  used  to  construct  them) 


class  make  dozens  of  different 
buttons,  and  esich  button  rnifjht  have 
its  own  color,  size,  label,  and  so  on. 


A  claw  Is  a  bhirOrini  for  anjjbjeCL  [t  iclls  the 
virtual  machine  h&vt  to  make  an 
particular  type, 
class  can  have 


(a)  Underlined  important  OOP  concepts 


Cass  and  Object 


Hey  Concepts 

*  D  bje  cts  are  m  atfe  up  of  the  data  (States]  a  tt  d  behaviors, 

*  Classes  a  re  made  u  p  o  f  attr  i  b  utes  ami  itiethods. 

*  Object  is  the  i  r  stan  cc  o  f  a  class.  Class  i  s  th  e  tempi  ate  of  object. 


(b)  Summaries  of  key  concepts  integrated  in  the  peer-to- 
peer  pre-reading  materials 


Encapsulation  and  Data  Hiding 

One  of  die  primary  adviiungw  of  using  object*  is  dm  the  object  need  riot  reveal  ail  ib  at¬ 
tribute*  and  behaviors.  In  good  DO  design  (at  least  what  i*  gcnenDy  accepted  as  gtHxlJ.aJ) 
object  should  only  reveal  the  interfaces  itut  other  objects  mutt  have  to  internet  with  it. 
Detail*  not  pertinent  to  tltt  uve  of  the  object  should  be  hidden  ho  in  all  other  objects. 

HEieapsularioii  is  defined  by  the  fact  that  objects  contain  both  the  attributes  ami  behav¬ 
iors,  Data  hiding  is  a  major  part  of  encapsulation. 

For  example,  an  object  that  calculates  the  square  of  a  number  must  provide  an  inter¬ 
face  to  obtain  the  result.  How 'ever,  the  internal  artribuccs  and  algorithms  u*cd  to  calculate 
i be  square  nad  not  be  nutle  available  to  the  requesting  object  Robust  classes  are  de¬ 
signed  with  encapsulation  in  mind.  In  the  next  suctions,  wc  cover  die  concepts  of  inter¬ 
face  anti  implementation,  which  ate  the  bails  of  encapsulaiion. 

Interfaces 

We  have  seen  that  die  interface  defines  the  fundamental  means  of  cuminuiifcatiun  be¬ 
tween  objects,  Each  elau  dtoign  specifics  the  interfaces  lot  the  proper  insemriation  and 
operation  ofribjrrt*.  Any  behavior  that  the  object  provides  rauw  be  invoked  by  a  mrcugc 
sent  using  one  of  the  provided  interface  vThc  interface  should  completely  describe  hove 
uvers  of  the  c!a»  interact  with  the  class.  In  most  OQ  languages,  die  methods  that  ate  port 
of  the  interface  arc  designated  as  public. 


(c)  Theoretical  explanations 


Figure  6.  Exemplar  excerpts  of  pre-reading  materials  (Sierra,  2005;  Weisfeld,  2009} 


Exercise  Time! 

1.  Find  3  devices/systems  that  potentially  use  the  concept  of  objects  and  classes 

2.  Find  3  websites  that  describe  the  difference  between  object  and  class. 


Figure  7.  Pre-reading  materials:  examples  of  embedded,  practical  exercises 


Encapsulating  the 
froodPog  class 


Z  .3  S  5  'i  .X, :  j  1  ii  OG  ■ 


GotxJDog 


X7 


pri  ■.  i  r  e 


P“fcl-.c  ir.t  net  Size  () 


getStzei } 
1  setSizc- ) 
!  &a*() 


t 


EX  -Ko-aK  tke  >*c-fcVo8i  d*"’v  realty 
X  „t-*  tV,w) 

,i  tWt  X  can  cKae--,;  Y«*r  -’»« 
Ijijr  vcm  i1,*-  Csm*e  W*  a-d  -a1*?  a 

XFod  iafe*-,  fa*ler.  MUt- 


Any  place  where  a 
particular  value  can 
be  used,  a  method 
call  that  returns  that 
type  can  be  used. 

instead  of: 

irrt  x  z  3  +  24; 

you  can  say; 

irrt  x  "  3  +  cne,getSi;zeQ; 


void  h<* r  k !  r  t 

it  i^izc  >  inj  * 

System. cue  .print In  v^wnoof  I  urooof !  ^ . 
1  il  >  14M 

Syy  print  !  Rl.  Ill*')  ; 

\  fr  1  f 

3  y  ^  cx  *  out .  p  C  i  -  i  1 1  ri  { taY  ip  1  Y  p  !  *  )  ; 


} 


-  n A 5  HOC d I « ."£ T&tfT  ID !' v &  { 

putU _ c  static  void  jr.-un  [SLrin^l  I  [ 

taotiCdg  one  =  new  CcedPc^O; 
or.t.x  set  Size  (73J  ; 

GoC'dDoT  T :wc  =  csv  G^adDe^O  ; 
two . S 1  ze  (0 ;i 

Syjs  L*i:n .  clC  .  piir.  tx;i  C  ^Dog  :  "  +  cr.B.gsLSize  (]  )  ; 

^ y ?■  r  !'m  _  a  i:  r  .  pr  i.  r.  1 1  n  J  r/wc  :  u  +  two  _  g$  ts  i  ze  £ )  )  ; 

rr. ft -hark  (l  ; 


1 


J 


Figure  8.  Pre-reading  materiafs:  sample  codes  expressing  OOP  concepts  (Sierra,  2005) 


Peer-to-Peer  Interactive  Discussions.  On  the  first  day,  the  first  interactive  discussion  section 
pairs  the  students  as  two-person  teams.  The  two-person  teams  interact  with  real-world  devices 
and  creations,  i.e.,  show-n-teli,  relating  the  topics  of  the  course  to  everyday  life.  The  instructor 
leads  the  discussion  on  how  programming  /  algorithms  play  a  role  in  devices  and  creations  with 


the  groups  for  15  minutes.  The  devices  and  creations  include  phones,  mobile  apps,  electronic 
devices  and  web  programs  on  the  Internet. 

The  second  interactive  discussion  section  is  the  daily  topic  discussion.  It  consists  of  two  parts. 
During  the  first  part,  the  instructor  spends  1 5  minutes  introducing  the  software  design  process  in 
general  according  to  personal  experience.  Subsequently,  the  instructor  introduces  a  design 
problem  and  its  settings  (designette).  Collaboratively  with  the  class,  the  instructor  decomposes 
the  design  problem  into  five  modules.  These  five  modules  are  to  be  used  in  the  Lab  sections 
everyday  -  two  modules  on  the  first  two  days  and  one  on  the  third  day. 

The  second  part  of  the  topic  discussion  is  a  peer-to-peer  interactive  discussion  on  the  first  days’ 
course  content:  Object  Oriented  Concepts.  In  the  first  30  minutes,  students  share  what  they  leam 
from  the  pre-reading  materials  in  groups  and  to  the  class  (Fig.  9).  In  the  next  45  minutes,  the 
instructor  presents  a  personal  understanding  on  the  same  set  of  material  with  the  aid  of 
PowerPoint  slides,  with  ad  hoc  examples,  that  are  programmed  interactively  and  collaboratively 
with  the  students  (Fig.  10). 


Figure  9.  Peer-to-Peer  interactive  discussion  and  class  presentation 


*■  Jemoristrace  ^te^ace  3nd  irplefiien-a-ivn 
*/ 

Erployea  employeel  =  new  Teacher( "Justin",  "Male”); 
erployeel.workf ! ; 

erpioyeel  =  new  Musician!" Simon",  “Male")} 
erployeel .work() ; 

/*># 

*  Output:. 

*  “eacn  Java 
“  Play  r.ir.  ic 

V 


Jiscussion;  the  object  employ  eel  ..  the  ’Contract"  to  the 
“  classes  that  implements  The  interface  tfta.&yre. 

'*  As  trie  type  changes  {"•"ori  Teacher  to  Musician),  the  actual 
imp linen tat  ion  changes  {“each  Java  to  Play  Music. 

*  do  we ve ' ,  the  "contract"  doesn't  chance  (work! 

*/ 


/  -V 

*  rrra  n  s  t  r  -3 1 e  a  bs  t  ra  c  t  io  n 
*/ 

Teacher  newTeacher  *  new  Teacherf ,l  Justin",  "Male'1); 

Musician  newMusician  =  new  Musician{”S_Lfr£fT f  "Male"); 

//  Teache-1.-:  state?  and  property 

System,  out-  println(41\nTeacher;  \n  My  narr e  is  ,J  +  newTeacher»naine 
+  h.  I'l  a  "  +  newTeacher. gender  +  11  11 
+  newTeactier.  getOccupation  ( )  + 
rewTe acner.worki ) ; 

/'/  Musician *>  states  and  property 

System  out  *  p  rim  ln("\nHusiciani  \n  My  name  is  11  -+  newMusician.^ame 
■+  ,r.  1’ir  a  ,J  +  newMusician. gender  +  11  ,l 
-*■  newMjsician-getQccupationO  +  "*")j 
ne*-jMusicianpWork[ ) ; 

/  *■ 

•“  Output-  ~eacher;  My  name  is  Justin.  IV  a  Male  Teacher.  Teach  Java 

*  Musician:  My  name  is  Simon.  IV  a  Male  Musician.  Play  rusic 
*/ 

//  Jis c-jssicn ;  Noticec  that  the  attributes  and  method  are  declared  in 
//  the  p-a re r z  type  (pe-son) 


Figure  10.  Instructor  sample  code  developed  and  used  during  the  interactive  discussion  sessions 


Collaborative  DBL  Lab .  The  collaborative  lab  sessions  are  driven  by  a  design  problem,  i.e., 
design-based  learning  via  designettes  (Wood,  2012).  As  an  example  designette  and  collaborative 


lab  experience,  students  work  in  groups  of  two  persons  to  develop  a  Fab  Lab  Management 
System  to  manage  booking,  training  and  schedule  viewing  of  various  equipment,  processes  and 
facilities  as  part  of  a  university  prototyping  shop.  Below  is  a  detailed  description  of  the 
designette  provided  to  the  students  as  part  of  the  experimental  study. 

Design  Problem  -  Fab  Lab  Management  System  (TLMSL 

A  software  system  is  needed  with  the  following  characteristics.  All  characteristics  described 
below  must  be  included  in  the  Fab  Lab  Management  software  system. 

System  Description: 

I  he  Fab  Lab  Management  System  (FLMS)  is  to  be  designed  as  part  of  the  lab  safety 
authentication.  It  automates  the  process  of  authenticating  students’  access  to  the  lab  and  its 
machines  and  processes.  In  order  to  gain  the  access,  students  will  need  to  use  FLMS  to  complete 
a  safety  test.  By  obtaining  the  safety  certificate,  students  are  then  able  to  use  the  other  features. 

The  usage  or  each  machine  or  process  in  the  lab  is  also  restricted  by  licenses.  Students  will  have 
to  attend  training  sessions  and  complete  the  certificate  tests  to  obtain  various  licenses  in  order  to 
use  the  respective  machines.  Using  FLMS,  students  can  view  the  training  schedule  and  make 
appointments  subjected  to  availability. 

Once  the  licenses  are  obtained,  students  can  activate  the  respective  machines  by  scanning  the 

student  identification  card  on  the  card  reader  attached  the  machine. 

Key  Features: 

1.  System  Architecture:  Ihe  system  architecture  of  FLMS  allows  effective  communication 
among  various  entities  such  as  user,  certificate,  and  appointment.  Each  of  these  entities  has 
concrete  entities,  such  as  student  and  admin  are  derived  entities  of  user,  and  safety  certificate  and 
various  machine  licenses  are  entities  of  certificate. 

2.  Log  in:  The  system  requires  users  to  log  in  before  using  any  other  features.  The  users 
(student/admin)  log  in  with  their  id  and  password.  The  system  is  able  to  distinguish  between 
students  and  admin,  and  direct  them  to  respective  pages. 

3.  Certificate  Test:  The  system  allows  students  to  check  and  book  slots  for  training  and 
examination  of  the  certificates.  Upon  passing  a  test,  students  will  obtain  the  respective  license. 
With  the  license,  students  can  then  activate  the  machine  by  scanning  the  student  identification 
card  on  the  card  reader  attached  to  each  machine.  In  this  process,  the  student’s  id  number  will  be 
passed  to  the  system  to  check  if  the  license  is  already  obtained  to  operate  the  machine. 

4.  Simulation  on  Machine  Activation:  Using  the  identification  number  and  password,  students 
can  activate  the  machines  in  the  Fab  Lab.  During  authentication,  the  system  checks  if  the  student 
has  obtained  the  license  of  the  machine.  Once  the  authentication  passes,  the  students  will  be 
given  30  minutes  to  run  the  machine. 

5.  Machine  Booking:  The  system  also  allows  the  student  to  check  the  calendar  for  the 
availability  of  the  machines,  and  book  a  time  slot  to  use  the  available  machines.  (More 
descriptions  may  be  written.  These  may  be  used  for  parallel  programming  topic.) 

6.  Data  Centre:  The  system  has  an  entity  to  store  the  data  which  is  shared  among  all  other 
entities.  Common  Data  such  as  user  info,  machine  info  and  machine  schedules  will  be  stored  in 


the  data  center.  The  data  center  does  not  allow  any  other  entity  to  create  a  copy  or  instantiate  it  at 
any  point  of  time.  The  data  will  not  be  erased  by  system  restart. 


p a c ka c e  J -a v aC lasses; 
irpcrr  java. util. List; 

public  class  Student  exce-tcs  User  { 


t  ( 

J  / 


TD 


Oeclaar  necessary  variables 


puslic  List<Certif icate>  getCert if icateList-J )  { 
return  certif icateList; 

> 

puc lie  void  addCe rt if icate( Certificate  certificate)  { 

//  "0  do:  Add  the  certificate  to  Data  Centre 

> 

puc tic  Student (St ring  id,  String  password)  { 
super ( id, password ); 
tniu. type  =  UserType. STUDENT; 


Figure  1 1 .  Example  pseudo-code  shared  with  student  peer-to-peer  teams  as  part  of  collaborative  labs 


There  are  in  total  five  lab  sessions,  two  sessions  each  on  the  first  two  days  and  one  on  the  last 
day.  For  each  lab  session,  students  work  on  one  module  of  the  design  problem.  The  level  ot 
complexity  and  difficulty  increases  gradually.  In  Collaborative  Lab  1,  students  are  to  complete 
the  first  module  of  the  design  problem,  which  is  discussed  in  the  morning.  Students  are  given  the 
basic  algorithm,  a  portion  of  pseudo-code  (Fig.  11)  and  a  test  case.  In  order  to  complete  the 
problem,  it  requires  the  students  to  complete  missing  portions  of  provided  code  and  run  the  test 
case.  Collaborative  Lab  2  entails  work  on  the  second  module  of  the  designette.  The  instructor 
introduces  and  describes  an  algorithm.  Students  develop  the  code  for  the  complete  code  for  the 
algorithm  and  run  the  given  test  case  to  validate  the  code.  In  Collaborative  Lab  3,  students  are 
given  an  algorithm.  They  are  expected  to  write  the  complete  code  and  test  case  to  solve  the 
problem  (design  module)  and  validate  the  code.  Collaborative  Lab  4  entails  work  on  the  fourth 
module  of  the  design  problem.  Students  are  to  design  the  algorithm  to  address  the  problem. 
Students  then  develop  complete  code  for  the  module  and  write  the  test  case  to  validate  the 
algorithm  and  code.  Collaborative  Lab  5  entails  work  on  the  fifth  module  of  the  designette. 
Students  are  to  design  the  algorithm  to  address  the  problem.  Subsequently,  students  develop  two 
alternative  codes  for  the  algorithm.  Lastly,  students  write  the  test  cases  to  validate  the  algorithm 
and  codes. 


Final  Epitome  Assignment.  On  the  afternoon  of  the  third  day,  both  the  control  and  experimental 
groups  are  given  a  designette  project  as  the  final  assignment,  A  real  customer  is  invited  to 
present  the  design  problem  separately  in  both  classes  (Fig.  12.2).  The  problem  corresponds  to  a 
real  situation  where  students  of  an  institution  need  to  be  sorted  into  “balanced”  teams  (according 
to  a  given  set  of  requirements)  to  improve  their  expected  results  and  performance.  The 
presentation  includes  background  knowledge,  design  requirements  and  expected  outcome.  The 
students  are  expected  to  design  an  algorithm  and  associated  software  for  effective  design  group 
formation  based  on  a  set  of  requirements,  including  the  team  members’  personality  type  (Jensen, 
2000),  working  style,  skill  set,  gender  and  nationality.  The  resulting  software  is  intended  to  be 
used  to  systematically  create  design  teams  of  a  predetermined  size  range  for  university  courses  of 
co-curricular  activities,  as  shown  in  Fig.  12.1. 
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Figure  12.1.  input  file,  requirements  and  expected  output 


The  workshop  students  receive  a  file  with  the  information  recorded  in  the  table  shown  in  Fig  1, 
and  resulting  software  is  intended  to  systematically  create  design  teams  (blue  header  column)  of 
a  predetermined  size  range  (5-6)  for  university  courses  of  co-curricular  activities. 


Figure  12.2,  Dr.  Diana  Moreno  presenting  a  design  problem  to  the  control  and  experimental  groups 


Ihe  students  in  the  control  and  experimental  groups  are  assigned  to  design  teams  of  four  or  five 
persons  to  work  on  the  project  for  one  and  a  half  days  (half  a  day  on  the  third  day  and  full  day  on 
the  fourth  day).  Teams  are  formed  separately  for  both  classes.  During  this  period,  the  two 
instructors  are  available  in  answering  questions  via  email.  On  the  last  day,  the  students  are 
brought  back  together.  Each  design  team  presents  their  solution  to  the  design  problem  to  the 
customer  and  instructors  (Fig.  13). 


Upon  completion  of  the  final  epitome  design  project,  all  students  in  the  course  complete  a  post¬ 
test  on  the  object-oriented  programming  concepts,  which  is  identical  to  the  pre-test  taken  at  the 
eginning  of  the  courses.  All  participants  also  complete  a  learning  experience  self-assessment, 
fhese  learning  assessment  and  evaluation  instruments  provide  the  experimental  data  for  the 
study,  i.e.,  pre-  and  post-tests  of  the  course  content,  a  questionnaire  on  the  learning  experience  by 
both  groups,  and  key  demographic  information.  These  data  are  analyzed  statistically  to  measure 
the  learning  outcomes  of  key  concepts  when  comparing  the  control  and  experimental  groups. 
Self-assessment  ot  learning  styles  and  approaches  are  also  analyzed  across  the  sample  sizes  for 
both  groups  and  with  respect  to  the  pre-knowledge  and  demographics  of  the  participants. 


Figure  1 3.  Exemplar  student  presentation  of  the  epitome  design  project 


IV.  Assessment  Instruments,  Data  Collection,  and  Analysis 

IV.  1  Learning  Experience  Self-Assessment  —  Survey 

The  survey  (Appendix  B)  is  designed  as  a  key  quantitative  measurement  for  the  experimental 
study.  It  is  designed  to  enable  participants  to  rate  their  experiences  with  particular  classroom  and 
lab  experiences,  in  addition  to  the  overall  course.  The  self-assessment  includes  three  sections. 
Section  I  gathers  demographic  information  of  the  student  such  as  gender,  age,  academic  studies 
and  programming  experience.  Section  II  gathers  quantitative  rankings  on  each  course 
component  for  both  the  experimental  group  and  control  group.  Students  rank  the  lecture  sessions, 
interactive  discussions,  labs,  relevance  between  lessons  and  labs,  and  overall  experience.  The 
scale  is  on  the  following  basis:  to  what  extent  the  components  are  interesting,  motivating, 
inspiring,  engaging  and  difficult.  We  likewise  gather  participant’s  self-assessment  on  efforts, 
focus,  successfulness  and  number  of  hours  students  spend  outside  the  class.  Section  III  gathers 
participants’  qualitative  feedback  and  suggestions  for  course  improvement. 


TV. 2  Pre-  and  Post-Tests 

Pre-  and  post-tests,  and  their  comparison  of  results,  are  a  key  quantitative  measurement  of  this 
pedagogical  experiment.  Both  instructors  for  the  experimental  study  were  informed  of  the  key 
course  topics  based  on  the  concept  inventory.  As  described  above,  both  the  traditional  and 
active-DBL  frameworks  used  were  structured  to  focus  on  these  concepts.  However,  the  pre-  and 
post-tests,  designed  and  constructed  from  the  course  inventory,  were  not  disclosed  to  the 


instructors  until  after  the  course  was  completed.  This  process  seeks  to  avoid  participants  in  either 
course  from  receiving  any  hints  during  lectures  or  interactive  discussions,  which  may  affect  the 
accuracy  of  the  result  of  the  experimental  study  and  associated  statistical  analysis. 

While  designing  the  pre-  and  post-test,  the  intent  is  to  ensure  the  quality  of  the  questions,  the 
ability  of  the  questions  to  measure  the  participants’  understanding  of  key  concepts,  and  how  well 
the  participants  are  able  to  apply  the  concepts  in  problem  solving.  Hence,  all  questions  are  set 
based  on  the  Object-Oriented  Programming  concept  inventory.  Both  the  pre-  and  post-tests 
contain  35  multiple-choice  questions  with  only  one  correct  answer  per  question.  In  terms  of 
question  type,  the  questions  include  Theory  and  Application  categories.  In  terms  of  difficulty,  the 
assessment  instruments  consist  of  Basic  Level  and  Advanced  Level  questions.  There  are  thus 
four  sub-types  of  questions,  namely  Basic  Theory,  Advanced  Theory,  Basic  Application,  and 
Advanced  Application.  By  comparing  both  the  overall  categories  and  sub-categories,  we  can 
assess  the  participants’  learning  of  key  concepts  and  clearly  measure  the  advantages  and 
disadvantages  between  the  traditional  pedagogy  and  the  active-DBL  approach. 

The  pre-test  was  completed  by  participants  in  the  briefing  before  the  actual  course  started.  This 
pre-test  serves  as  an  indicator  and  reference  of  a  participant’s  programming  background  and 
prior  knowledge  and  understanding  in  OOP.  On  the  other  hand,  the  post-test  was  completed  on 
the  last  day  of  the  course,  after  the  students  had  completed  all  lessons  and  assignments.  The  post¬ 
test  allows  us  to  measure  how  well  the  students  have  learned  the  key  concepts  and  provide  a 
comparison  with  pre-test  conditions. 

Both  the  pre-  and  post-tests  are  designed  to  be  identical.  By  doing  so,  we  aim  to  obtain  minimal 
error  in  measuring  the  quality,  effectiveness  and  efficiency  of  both  classes.  Figure  15  shows  pre- 
and  post-test  sample  questions,  and  Appendix  A  provides  a  complete  pre-/post-test  as  a  concept 
inventory.  The  overall  difficulty  of  the  assessment  instruments  is  judged  to  be  relatively  high. 
Each  participant  student  is  given  45  minutes  to  complete  the  test.  In  order  to  choose  the  correct 
answer,  the  students  must  folly  understand  the  tested  concepts.  In  addition,  every  question  has  4 
-  8  options  with  only  one  correct  answer.  This  design  aims  to  avoid  students  from  scoring 
through  guessing. 


Basic 

Theory 

(BT) 

What  is  the  appropriate  data  type  for  this  value:  "Dog"? 

A*  ant 

B.  String 

C  double 

D.  boolean 

E.  i  don't  know 

Advanced 

Theory 

(AT) 

Which  of  the  following  statement  is  not  one  of  the  restrictions  In  each  enum  declaration? 

A.  Enum  constants  are  implicitly  final,  because  they  declare  constants  that  shouldn't  be  modified, 

B-  Enum  constants  are  implicitly  static, 

C.  Any  attempt  to  create  an  object  of  an  enum  type  with  operator  "new"  results  in  a  compilation  error. 

D.  The  keyword  static  must  be  present  while  declaring  the  fields  of  an  enum, 

E.  1  don't  know. 

Basic 
Applicati 
on  (BA) 

Which  of  the  following  piece  of  code  will  compile? 

A.  Ar  ray  Li  st<  N  u  m  be  r>  numberlist  =  new  Array  Ust<integer>(); 

B.  ArrayU st<?  Extends  Number>  numberlist  =  new  ArrayUst<lnteger>(); 

C  Array  List  <?  super  Number>  numberlist  -  new  ArrayUst<lnteger>Q; 

D.  ArrayList<Number>  numberlist  ^  Arrays.asUst(l,2,3A5); 

E.  None  of  the  above, 

F.  f  don't  know. 

Advanced 
Applicati 
on  (AA) 

public  class  MyCiassl  { 
public  void  ml(int  i)  {} 
public  void  m2(int  i){} 
public  static  void  m3{int  i)  {} 
public  static  void  m4(int  1}  {} 

} 

public  class  MyClass2  extends  MyCiassl  { 
public  static  void  i}{) 

public  void  m2{int  i)  {} 
public  void  m3(int  i)  {} 
pubiic  static  void  m4{int  i)  {} 

) 

Which  method  hides  a  method  in  the  superclass? 

A.  ml 

B.  m2 

C.  m3 

D.  m4 

E.  All  of  the  above 

F.  None  of  the  above 

G.  1  don't  know 

Figure  15.  Pre-VPast-Test  sample  questions 


IV.  3  Assessment  Data  and  Analysis 

There  are  two  key  assessments  and  evaluations  of  the  experimental  study.  The  first  is  the 
learning  self-assessment  survey.  The  other  is  participant  performance  on  the  pre-  and  post-tests. 

The  survey  aims  to  measure  the  qualitative  self-assessment  and  self-reported  motivation  for 
learning  fundamental  programming  and  software  engineering  principles.  All  questions  expect 
single  answers.  For  all  questions  except  6.1,  there  are  five  categories  of  answers,  ranging 
between  two  extremes.  For  coding  purposes,  we  map  a  Likert  scale  to  the  responses, 
corresponding  to  -2  to  2  to  represent  the  assessment  score.  Question  6.1  asks  participants  about 
the  number  of  hours  spent  in  independent  study  after  class.  For  this  question,  we  use  numbers 
from  0-8  to  represent  the  answers. 


A  power  analysis  for  sample  size  shows  a  minimum  sample  size  of  approximately  eight 
participants  per  group  to  obtain  a  power  of  80%  for  a  difference  in  survey  scores  between  0  5 
and  1.0  and  an  expected  standard  deviation  of  0.5.  Of  course,  a  higher  sample  size  is  desired  to 
assure  this  power  level  and  avoid  significant  biases  that  may  exist  due  to  such  factors  as 
participant  background. 

Based  on  the  voluntary  enrollment  during  IAP,  there  are  a  total  of  16  actual  participants  (out  of 
29  registered  participants)  that  attended  all  lessons,  completed  the  survey  and  both  tests.  Nine  of 
the  participants  are  in  die  experimental  group,  whereas  seven  are  in  the  control  group.  The  data 
collected  for  the  experimental  study  are  summarized  in  Appendix  C. 

The  data  is  analyzed  by  comparing  the  experimental  and  control  groups  using  a  student-t,  two- 
tail  test.  The  average  scores  and  standard  deviations  are  calculated  for  both  the  experimental  and 
control  groups,  followed  by  the  statistical  test.  Table  2  summarizes  the  results  of  this  analysis. 
Two  of  the  questions  have  clear  statistical  significance  with  p-values  less  than  0.05.  One 
question  is  m  the  p- value  range  of  0.05  to  0.10.  And  two  questions  have  p-values  very  near  0  10 

All  other  questions  may  be  used  to  analyze  common  trends,  when  agreement  exists  in  positive  or 
negative  Likert  scores. 


Table  2.  Statistical  Analysis  of  Learning  Self-Assessment  Survey  Results 


- - - - — _ _ _ _ 

Question 

Average 

Compare 

Exp 

Ctrl 

Diff 

STD  Exp 

STD  Ctrl 

ttest 

Ql.l 

Lecture/Discussion:  Uninteresting  /  Interesting 

1*13 

1.14 

-0.02 

0.64 

0.38 

0.7686 

QL2 

Lecture/Discussion:  De-Motivating  /  Motivating 

0.50 

0.71 

-0.21 

0*53 

0.76 

0*6459 

Q1.3 

Lecture/Discussion:  Frustrating/  Inspiring 

0.63 

0.71 

-0.09 

0.52 

0*95 

0.9071 

Q1.4 

Lecture/Discussion:  Unengaging  /  Engaging 

1*38 

1.14 

0.23 

0.52 

0.69 

0.5513 

Ql.S 

Lecture/Discussion:  Difficult  /  Easy 

0.13 

0,00 

0.13 

0.64 

0.82 

0.4440 

Q2.1 

Lab:  Uninteresting/ Interesting 

1.00 

0.57 

0.43 

0*53 

0.53 

0.1265 

Q2.2 

Ub:  De-motivatmg  /  Motivating 

0.63 

0.43 

0.20 

0.52 

0.79 

0.5014 

Q2.3 

Lab:  Frustrating  /  Inspiring 

0.25 

0.14 

o.u 

0.71 

0*69 

0.5970 

02*4 

Lab:  Unengaging  /  Engaging 

0.88 

-0.14 

1*02 

0.35 

0*90 

0.0148 

Q2.S 

Lab:  Difficult  /  Easy 

-0.63 

-0.29 

-0.34 

0.92 

0.49 

0.6874 

03.1 

Lecture  with  Lab:  Irra  levant/  Closely  relevant 

1.13 

0.29 

0.84 

0.64 

1,11 

0.0795 

03*2 

Lecture  with  Lab:  Unhelpful  /  Helpful 

1*13 

0.57 

0.55 

0*64 

0.98 

0*1612 

Q4.1 

$elf~accessment:  Min  effort  -  max  effort 

0.63 

1*14 

-0*52 

0.52 

0.69 

0*1533 

04*2 

fseif-accessment:  Unfocussed  -  Focus 

0.38 

1.00 

-0.63 

0-74 

0.58 

0*1104 

043 

Self-accessment:  Unsuccessful  -  Successful 

0*00; 

1.14 

-1*14 

0.76 

0.38 

0.0046 

Q5.1 

Time  vs  Work 

1*00 

0.71 

6.29 

0.00 

0.49 

0.4360 

Q6.1 

Hr  spent  outside  classroom 

4.63 

3.57 

1.05 

1.92 

1*27 

0.5696" 

Q7.1 

Overall:  Uninteresting  /  Interesting 

1.25 

1.14 

0*11 

0.46 

0.38 

0.7045 

Q7.2 

Overall:  Demotivating  /  Motivating 

0.75 

0.86 

-o.u 

0.46 

0.90 

0.6273 

Q73 

Overall:  Frustrating  /  Inspiring 

0.88 

0.71 

0.16 

0.64 

0.76 

0.6264 ; 

07.4 

Overall:  Unengaging  /  Engaging 

1.00 

1*00 

0*00 

0*53 

0,58 

1.0000 

Q7.5 

Overall:  Difficult  /  Easy  | o  13 

0.00 

-0.13 

0.35 

0.82 

0,7877 

Prer/Post-Test  In  both  ol  the  pre-  and  post-tests,  all  questions  have  only  one  correct  answer.  In 
coding  the  responses,  a  zero  (0)  is  used  to  represent  a  wrong  answer  and  a  one  (1)  is  used  to 


represent  a  correct  answer.  The  pre-test  scores  and  the  post-test  data  are  listed  in  Appendix  D. 
Because  of  a  lower  power  for  sample  size  the  pre-  and  post-test,  and  because  of  no  valid 
statistical  significance  between  the  experimental  and  control  groups,  only  raw  data  are  chosen  for 
the  answers  to  concept  inventory  questions,  in  addition  to  the  differences  in  pre-  and  post-test 
performance  across  participants.  The  distributions  of  performance  on  the  pre-  and  post-tests, 
listed  in  the  “Sum”  rows  and  summarized  as  means  and  standard  deviations,  are  compared  across 
all  participants,  as  shown  in  Table  3,  using  a  two-tailed  student-t  test.  Statistical  significance 
between  the  pre-  and  post-test  performance  exists,  where  the  p-value  is  0.00000132,  i.e.,  « 
0.01. 

Table  3.  Statistical  analysis  of  performance  difference  across  participants 


Mean 

Standard  Diviation 

Student-t 

Pre-test 

Post-test 

Pre-test 

Post-test 

15,69 

22.44 

3.24 

3.10 

0.00000132 

V.  Discussion  and  Conclusions 

Section  IV  presents  the  assessment  and  evaluation  instruments  for  the  experimental  study,  the 
raw  data  collected  with  these  instruments,  and  a  basic  statistical  analyses  of  the  results. 
Considering,  first,  the  participants’  self-assessment  of  learning,  the  experimental-study  results 
show  a  range  of  implications.  Based  on  the  survey  data,  we  conclude  as  follows  (referring  to 
Tables  2): 

For  Interactive  Discussion  /  Lecture  sessions:  (i)  both  groups  find  these  sessions  of  the  course 
capture  their  interest;  (ii)  both  groups  are  motivated  within  these  sessions  of  the  course,  on 
average,  with  a  higher  variance  in  the  control  group;  (iii)  both  groups  are  inspired,  on  average, 
with  a  higher  variance  for  the  control  group;  (iv)  both  groups  find  these  sessions  to  be  engaging, 
and  (v)  both  groups  are  neutral  regarding  the  level  of  difficulty,  i.e.,  the  level  of  difficulty  is 
neither  too  high  or  too  low. 

For  Lab  sessions:  (i)  the  experimental  group  finds  the  lab  sessions  to  be  more  interesting;  (ii) 
both  groups  are  positive  about  the  lab  sessions  in  general;  (iii)  both  groups  are  relatively  positive 
in  finding  the  lab  sessions  to  be  inspiring;  (iv)  the  experimental  group  has  a  clearly  engaging  lab 
experience,  greater  than  that  of  the  control  group;  and  (v)  both  groups  find  the  labs  to  be 
difficult,  with  higher  variance  by  the  active  learning  group. 

For  relevance  between  Interactive  Discussion  /  Lecture  and  Lab:  (i)  both  groups  are  indicate  that 
the  sessions  show  relevant  connectivity,  with  the  active  learning  group  having  a  significantly 
higher  rating  and  the  control  group  having  a  higher  variation;  (ii)  both  groups  find  the  sessions  to 
be  helpful,  with  the  experimental  group  having  higher  rating  and  the  control  group  having  higher 
variation;  (iii)  both  groups  devoted  significant,  but  with  the  control  group  having  higher 
indicated  effort. 

For  Overall  Self-Assessment:  (i)  both  groups  exhibited  focus,  but  with  higher  indicated  focus  by 
the  control  group;  (ii)  the  control  group  indicates  more  personal  success,  where  the  focus  is  on 
homework  and  homework-like  labs  as  opposed  to  more  open-ended  design-based  labs;  (iii)  both 
groups  feel  challenged  by  quantity  of  work  for  the  given  time;  (iv)  the  experimental  group,  on 
average,  dedicates  more  time  to  the  sessions,  but  there  does  not  exist  a  statistically  significant 


difference;  and  (v)  both  groups  fmd  the  course  very  interesting;  (vi)  both  groups  find  the  course 
motivating;  (vii)  both  groups  find  the  course  inspiring;  (viii)  both  fmd  the  course  engaging;  and 
(ix)  both  groups  fmd  the  course  engaging. 

Overall,  based  on  a  self-reported  assessment,  the  active  learning  -  DBL  group  finds  the  course 
elements  to  be  more  interesting,  engaging,  and  with  much  higher  connectivity  than  the  control 
group.  However,  even  with  these  clear  distinctions,  the  control  group  feels  they  achieve  more 
personal  success  than  the  active-learning  group.  We  interpret  that  their  response  may  be  based  on 
their  experience  with  a  more  homework-based  course  content,  or,  alternatively,  active  learning 
was  potentially  more  challenging,  especially  the  design-based  laboratories  and  peer-to-peer 
interactive  discussions.  Both  groups  find  the  overall  experience  to  be  interesting,  motivating, 
inspiring  and  engaging,  indicating  the  careful  design  of  each  pedagogical  approach. 

In  addition  to  the  self-assessment  surveys,  the  pre-  and  post-test  results  show  a  clear  finding 
(Tables  4-6).  Even  though  there  exists  no  statistically  significant  differences  in  performance 
between  the  experimental  and  control  groups,  both  groups  showed  a  marked  improvement  in 
learning  the  course  content.  This  result,  as  measured  by  a  systematically  developed  concept 
inventory,  is  reasonable  for  a  short  course  with  purely  volunteer  participants.  But  it  also 
provides  credence  to  the  experimental  findings.  Because  of  the  distinct  learning  by  both  groups, 
the  experimental  study  is  not  biased  by  poorly  designed  course  content  or  course  implementation 
frameworks.  Instead,  participants  learned  a  range  of  desired  concepts,  where  the  implication  is 
that  either  course  framework  could  be  used  in  future  software  engineering  courses,  but  with 
higher  likelihood  of  successful  in  student  performance  for  integrated  active  learning  and  DBL 
due  to  the  higher  degree  of  interest,  engagement,  and  connectivity  perceived  by  the  participants 
through  various  course  elements. 

Contributions.  Returning  to  the  stated  hypothesis  of  the  experimental  study,  i.e.,  “ There  exists 
significant  improvement  in  the  engagement,  student  interest,  and  motivation  for  software 
engineering  content  using  an  integrated  approach  of  active  and  deign-based  learning 
compared  to  traditional  teaching  approaches ,”  the  experiment  results  provide  clear  positive 
support  for  this  hypothesis.  Thus,  a  key  contribution  of  this  paper  is  the  potential  shown  for 
combining  active-learning  and  DBL  approaches,  especially  in  adapting  and  extending  recent 
engineering  education  research  findings,  such  as  designettes.  This  contribution  extends  to  the 
particular  elements  developed  for  object-oriented  programming  courses,  and,  more  generally, 
software  engineering.  Other  significant  contributions  of  the  study  include  the  development  of  an 
initial  concept  inventory  for  object-oriented  programming  concepts,  a  self-assessment  instrument 
for  student  learning,  and  course  frameworks  demonstrating  a  spectrum  for  teaching  engineering 
subject  matter. 

Limitations  and  Future  Work .  While  the  results  of  the  experimental  study  are  very  encouraging 
regarding  active  learning  and  design-based  learning,  there  are  a  number  of  limitations  in  the 
current  study.  These  limitations  include  a  relatively  low  sample  size,  especially  for  the  pre-  and 
post-test  data;  evolutions  and  validation  needed  for  the  concept  inventory;  the  use  of  different 
instructors  to  teach  courses  for  the  control  versus  experimental  groups;  and  the  use  of  a  short- 
course  format  and  setting  as  opposed  to  foil  term  courses.  Future  work  will  address  approaches 
to  overcome  these  limitations  and  may  include  a  hybrid  of  teaching  approaches,  higher  sample 
size  studies,  factor  studies  on  different  types  of  peer-to-peer  discussions  and  designettes,  and 
refinements  of  the  concept  inventory.  Moving  forward,  we  will  seek  opportunity  to  implement 
the  model  in  a  semester-long  course,  with  students  who  are  not  volunteers. 
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Appendix  A  -  Fre-/Post-test  concept  inventory 

The  questions  are  categorized  into  Basic/Advanced,  and  Theory/Application,  The  four  cross* 
categorizations  are  Basic  Theory  (BT)}  Basic  Application  (BA),  Advanced  Theory  (AT)  and  Advanced 
Application  (AA),  The  questions  are  in  randomized  order.  The  categorizations  are  not  disclosed  to  the 
students. 

I  .  [BT]  Choose  the  best  definition  of  a  Class. 

A.  A  special  type  of  object 

B.  A  group  definition,  containing  the  data  and  function  elements  necessary  to  create  an  object 

C.  An  action  for  a  program 

B  .  A  group  of  students  in  a  room 
E.  I  don’t  know 

2.  [BT]  Choose  the  best  definition  of  an  object 

A.  A  thing 

B.  An  instance  of  a  class 

C.  Something  you  wear 

D.  A  class 

E.  I  don’t  know 

3.  [BT]  What  is  the  appropriate  data  type  for  this  value:  “Dog” 

A.  int 

B.  String 

C.  double 

D.  boolean 

E.  I  don’t  know 

4.  [BT]  Which  of  the  following  item(s)  can  he  considered  object(s)? 

A.  A  circle 

B.  A  dog 


C*  A  siring  of  text 

D.  All  of  the  above 

E.  I  don't  know 

5.  [BT]  Which  of  the  following  statements  best  describes  the  relationship  between  an  object  and  a  class? 

A.  The  term  Object  is  used  to  describe  the  memory  allocation  whereas  class  is  used  to  describe  the 
data  structure.  Both  of  them  refer  to  an  identical  concept  in  Object-Oriented  Programming. 

B.  An  object  is  a  template  of  a  class  and  a  class  is  an  instance  of  an  object.  The  state  and  behaviors  of 
an  object  define  the  attributes  and  methods  of  a  class  respectively. 

C.  An  object  is  a  template  of  a  class  and  a  class  is  an  instance  of  an  object.  The  attributes  and 
methods  of  an  object  define  the  state  and  behaviors  of  a  class  respectively. 

D.  A  class  is  a  template  of  an  object  and  an  object  is  an  instance  of  a  class.  The  state  and  behaviors  of 
an  object  define  the  attributes  and  methods  of  a  class  respectively. 

E.  A  class  is  a  template  of  an  object  and  an  object  is  an  instance  of  a  class.  The  attributes  and 
methods  of  a  class  define  the  state  and  behaviors  of  an  object  respectively. 

K  1  don't  know 

6.  [A A]  In  Addendum  I,  what  is  the  output? 

A.  Nike  10  0123456789 

B.  Nike  II  0  I  23456789 

C.  Nike  10  0  10  20  30  40  50  60  70  80  90 

D.  Adidas  10  0  1  2345678  9 

E.  Adidas  1101234567  8  9 

F.  Adidas  10  0  10  20  30  40  50  60  70  80  90 

G.  Compilation  error 

H.  I  don't  know 

7.  [BT]  What  is  the  role  of  a  constructor? 

A.  Make  the  code  confusing 

B.  Create  an  instance  of  a  class 

C.  Create  names  for  methods 

D.  Create  some  type  of  change  in  the  state  of  an  object 

E.  I  don't  know 

8.  [BA]  In  Addendum  1 ,  which  of  the  classes  practiee(s)  encapsulation? 

A.  Ball 

B.  Basketball 

C.  Soccer 

D.  All  of  the  above 

E.  None  of  the  above 

F.  I  don't  know 

9.  [BT]  Which  of  the  following  item(s)  can  be  considered  object(s)? 

A.  A  boolean 

B.  A  dog 

C.  An  array  list  of  Integers 

D.  All  of  the  abo  ve 

E.  Two  of  the  above 

F.  None  of  the  above 

G.  I  don't  know 

10.  [AT]  What  is  essential  for  encapsulation? 

A.  It  exposes  of  the  data  in  order  to  allow  efficient  access  of  the  data.  The  common  practice  of 
encapsulation  is  to  mark  instance  variables  public  and  mark  getters  and  setters  private. 


B.  It  exposes  of  the  data  in  order  to  allow  efficient  access  of  the  data.  The  common  practice  of 
encapsulation  is  to  mark  instance  variables  private  and  mark  getters  and  setters  public. 

C  It  restricts  access  to  the  data  in  order  to  protect  and  prevent  unnecessary  modification  of  the  data. 
The  common  practice  of  encapsulation  Is  to  mark  instance  variables  public  and  mark  getters  and 
setters  private. 

D.  It  restricts  access  to  the  data  m  order  to  protect  and  prevent  unnecessary  modification  of  the  data. 
The  common  practice  of  encapsulation  is  to  mark  instance  variables  private  and  mark  getters  and 
setters  public. 

E.  I  don't  know 

1 1 .  [BT]  Which  of  the  following  statement  are  not  true? 

A.  Encapsulation  is  a  form  of  data  hiding, 

B.  Encapsulation  typically  allows  programs  to  run  faster 

C.  Encapsulation  helps  to  protect  data  from  corruption 

D.  Encapsulation  allows  for  changes  to  the  internal  design  of  a  class  while  the  public  interface 
remains  unchanged, 

E.  Encapsulation  usually  increases  the  size  of  the  code. 

F.  I  don’t  know 


12.  [BT]  Inheritance  allows  code 

A,  Reusability 

B.  Reliability 
C  Usefulness 
D.  Correctness 
£.  Readability 

F.  I  don’t  know 

13.  [BT]  A  java  subclass  can  be  defined  by  using  the  keyword: 

A.  public 

B.  private 

C.  final 

D.  extends 

E.  inherits 

F.  None  of  the  above 

G.  I  don’t  know 


14.  [AA]  Refer  to  Addendum  2,  which  method  overrides  a  method  in  the  superclass? 


A. 

ml 

B. 

m2 

C, 

m3 

D. 

m4 

E* 

All  of  the  above 

F. 

None  of  the  above 

G. 

I  don’t  know 

15.  [AA]  Refer  to  Addendum  2,  which  method  hides  a  method  in  the  superclass? 


A. 

ml 

B. 

m2 

C, 

m3 

D. 

m4 

E. 

All  of  the  above 

F, 

None  of  the  above 

G. 

I  don’t  know 

16.  [BA]  Refer  to  Addendum  1 ,  which  of  the  following  statement  is  true? 


A.  Ball  is  a  sub-class  of  Sport 

B .  B  aske  tb  ail  is  a  sub-  class  of  B  al  1 

C.  Soccer  is  a  sub-class  of  Basketball 

D.  Ball  Game  is  a  sub-class  of  Sport 

E.  I  don't  know 


17,  [BT]  Java  provides  an  approach  known  as _ as  a  convenient  alternative  to  implement  multiple 


inheritance: 

A, 

Sandbox 

Inheritance 

c 

Packages 

D. 

Interface 

E. 

Class 

F. 

None  of  the  above 

G, 

I  don’t  know 

IS.  [AT]  Which  of  the  following  statement  is  not  true? 

A-  Polymorphism  allows  the  reference  type  to  be  the  super  class  of  the  actual  object 

B.  Polymorphism  allows  you  to  introduce  new  sub  class  types  without  changing  the  original  code 

C.  Polymorphism  is  a  prerequisite  of  an  inheritance  hierarchy 

D.  With  Polymorphism,  anything  that  extends  the  declared  reference  variable  types  can  be  assigned 
to  the  reference  variable 

E.  None  of  the  above 

F.  I  don’t  know 

1  9 .  [AA]  Refer  to  Addendum  ls  which  of  the  following  statement  will  not  compile? 

A*  Ball  ball  =  new  Ball(“MyBrand”); 

B.  Basketball  basketball  =  new  BasketbaO(ctMy Brand”); 

C.  Soccer  soccer  =  new  Soccer(“My  Brand”); 

D.  Ball  ball  =  new  Basketball  (“MyBrand”); 

E.  Sport  sport  =  new  Basketballf'My  Brand”); 

F.  All  of  the  above 

G.  None  of  the  above 

H.  I  don't  know 


20*  [AT]  Where  is  Collection  interface  from? 


A. 

java.io 

B. 

javaJang 

a 

javamtil 

D. 

java.awt 

E, 

java.sql 

F. 

I  don’t  know 

21.  [BT]  A 

container  can 

A, 

Hold  water 

B. 

Hold  objects 

C. 

Change  the  behaviors  of  a  reference  type 

D+ 

I  don't  know 

22.  [BT]  Which  of  the  following  type  belongs  to  a  sub  type  of  Collection? 

A.  An  array  of  strings 

B.  ArrayList<String> 

C.  HashMapKString,  $tring> 

D.  HashtaMe 

E.  All  of  the  above 


F,  None  of  the  above 

G.  I  don't  know 


23.  [AT]  Consider  the  following  statements: 

•  Each  element  must  be  unique. 

•  Duplicate  elements  must  not  replace  old  elements. 

*  Elements  are  not  key/value  pairs. 

*  Accessing  an  element  can  be  almost  as  fast  as  performing  a  similar  operation  on  an  array. 

Which  of  these  classes  provides  the  specified  features? 

A*  LinkedList 

B.  TreeMap 
C*  TreeSet 

D.  HashMap 

E.  HashSet 

F.  LinkedHashMap 

G.  Hashtable 

H.  None  of  the  above 

I.  1  don’t  know 


24.  [AT]  Which  of  the  following  statement  is  true? 

A.  Set  allows  only  one  copy  of  each  object. 

B.  Set  has  an  auto-assigned  index  for  every  element. 

C.  Elements  in  a  TreeSet  are  sorted  in  descending  order. 

D.  Elements  in  a  HashSet  is  sorted  in  ascending  order. 

E.  None  of  the  above. 

F.  I  don't  know. 

25  .  [BT]  Which  of  the  following  code  practices  generics? 

A.  String[]  si  -  {“Hello”,  “World”}; 

B .  public  static  void  main(String[]  args)  { } 

C.  ArrayList<lneger>  intList  =  new  Array List<Integer>; 

D.  All  of  the  above 

E.  I  don't  know 


26.  [BT]  Which  of  the  following  statement  is  true? 

A.  The  use  of  generics  allows  you  to  specify  the  exact  types  of  objects  that  a  particular  data  structure 
will  store, 

B.  The  presence  of  generics  provides  the  means  to  create  general  models  of  methods  and  classes  that 
can  be  declared  once,  but  used  with  many  different  data  types. 

C.  The  generic  collections  are  backward  compatible  with  Java  code  that  was  written  before  generics 
were  introduced. 

D*  To  work  with  generics,  every  element  of  the  array  must  be  an  object  of  a  class  or  interface  type. 

E.  All  of  the  above, 

F.  I  don't  know 


27.  [BA]  Which  of  the  following  piece  of  code  will  compile? 

A.  ArrayList<Number>  numberList  =  new  ArrayList<Integer>(); 

B.  AirayList<?  Extends  Number>  numberList  =  new  ArrayList<Integer>(); 

C.  AirayListc?  super  Number>  numberList  -  new  Array  Li  $t<l  nteger>{) ; 

D.  ArrayList<Number>  numberList  =  Arrays,asLlst(l  ,2,3 ,4,5); 

E.  None  of  the  above. 


F.  I  don't  know. 


28.  [BT]  Enum  is: 

A.  A  special  word 

B.  A  special  int 

C  A  special  String 

D.  A  special  method 

E.  A  special  Class 

F.  I  don't  know 

29.  [AT]  Which  of  the  following  statement  is  true? 

A.  Enum  defines  a  set  of  constants  represented  as  unique  identifiers. 

B.  An  enum  type  is  a  reference  type. 

C.  Enum  allows  declaration  of  methods,  constructors  and  Fields  . 

D .  Ennm  can  have  a  main  method. 

E*  All  of  the  above. 

F.  None  of  the  above. 

G.  I  don't  know. 

30.  [AT]  Which  of  the  following  statement  is  not  one  of  the  restrictions  in  each  ennm  declaration? 

A.  Enum  constants  are  implicitly  final,  because  they  declare  constants  that  shouldn't  be  modified. 
R.  Enum  constants  are  implicitly  static. 

C.  Any  attempt  to  create  an  object  of  an  enum  type  with  operator  “new15  results  in  a  compilation 
error. 

D.  The  keyword  static  must  be  present  while  declaring  the  fields  of  an  enum. 

E.  I  don't  know. 


3 1 .  [AP]  Refer  to  Addendum  3.  What  is  the  result  of  compiling  and  running  the  following  code? 

A.  "It  is  a  color  type5’  once  followed  by  '"BLUE". 

B,  "It  is  a  color  type"  twice  followed  by  "BLUE". 

C.  "It  is  a  color  fype"  thrice  followed  by  "  BLUE". 

D.  "It  is  a  color  type"  four  times  followed  by  "BLUE". 

E,  Compilation  fails. 

F,  I  don't  know. 


32.  [RT]  TO 

A. 

B. 

C. 

D. 

E. 

F. 


in  Java  stands  for 

Inside  and  Outside 

Input  and  Output 

Industrial  and  Organizational 

Interest  Only 

All  of  the  above 

I  don't  know 


33.  [AT]  What  is  required  to  make  a  class  serializable? 

A.  Encapsulate  all  attributes 

B.  Make  all  methods  public  or  protected 
C  Implement  Serializable  interface 

D.  Explicitly  declare  SerialYersionUID 

E.  All  of  the  above 

F.  I  don't  know 

34.  [AT]  Which  of  the  following  statements  describes  a  feature  of  Java's  io? 

A.  Ability  to  export  objects  with  their  states  and  behaviors  into  a  plain  text  file 

B.  Provide  a  uniform  set  of  methods  that  accept  and  return  File  objects 


C*  Comparability 

D.  All  of  the  above 

E.  1  don’t  know 


35.  [BT]  Which  of  the  following  statements  is  true  about  XML 

A.  XML  was  designed  to  transport  and  store  data. 

B.  XML  tags  are  pre-defmed 

C.  XML  is  not  human-readable 

D.  XML  can  only  be  read  by  applications  that  are  developed  using  object-oriented  programming 
language. 

E.  All  of  the  above 

R  None  of  the  above 

G.  I  don’t  know 

Addendum  1 

public  interface  Sport  {void  playQ;} 

public  abstract  class  Ball  implements  Sport  { 
private  String  brand; 
public  String  getBrandQ  {return  brand;} 
public  void  setBrand(String  brand)  {this.brand  =  brand;} 
public  Ball(String  brand)  {this.brand  =  brand;} 
public  void  playQ  {System.owL printings  tart  playing!");} 

} 

public  class  Basketball  extends  Ball  { 

public  Basketball(S tring  brand)  {super(brand);} 
public  void  work()  {SystermowLprintln(fThrow!");} 

} 

public  class  Soccer  extends  Ball  { 

public  Soccer(String  brand)  {superfbrand);  } 
public  void  playQ  {System.owf,priEtln(TTKick!");  } 

} 

public  class  BallGame  { 

public  static  void  main(String[]  args)  { 

Ball  ballGame  =  new  BasketballQNike'1); 
int  numberOfPlayers  =  10; 
int[]  jerseyNumbers  =  {  0,  1?  2,  3?  A,  5,  6,  7,  8?  9  }; 
change$tat$(b&l\Gmi£y  numberO fPlay  ers  a  jerseyNumbers); 
System,owLprint(ballGame.getBrandO  +  "  "  +  numberOfPlayers  +  "  "); 

for(int  i :  jerseyNumbers) {System,oi/f.print(i  +  " ft);} 


} 

private  static  void  changeStats(Ball  firstGame,  int  numb  erOfP  layers, 
int[]  jerseyNumbers)  { 
firstGame  =  new  Soccer("  Adidas"); 
numberOfPlayers  =  11; 

for  (int  i  =  0;  i<jereeyNumbers.length;i++){jerseyNumbers[i]*=10;} 

} 

} 

Addendum  2 

public  class  MyClassl  { 
public  void  m  1  (int  i)  ( } 
public  void  m2(int  i)  {) 
public  static  void  m3  (int  i)  {} 
public  static  void  m4(mt  i)  {) 

} 

public  class  MyClass2  extends  MyClassl  { 
public  static  void  ml  (int  i)  {} 
public  void  m2(int  i)  {} 
public  void  m3  (int  i)  {} 
public  static  void  m4(int  i)  {} 

} 

Addendum  3 

enum  Color  { 

RED,  BLUE,  GREEN; 

private  AccountTypeO  (System.ow  r.printlnf'  1 1  is  a  color  type");} 

> 

class  TestEnum  { 

public  static  void  main(String[]  args)  { 

System, oiit.println(Color.^Lf/£}; 

} 


} 


Appendix  B  -  Survey 

The  survey  form  serves  as  the  learning  self-assessment  instrument  for  the  experimental  study. 


Survey 

Please  answer  the  following  survey .  Thank  you  far  your  hard  workl 
SECTION  I 


SECTION  II 

Read  each  sentence  and  please  enter  an  in  the  box  nearest  to  you1-  response 


L 

Please  indicate  your  gender. 

A.  Mate 
Female 

2. 

Please  Indicate  your  age. 

Answer: 

3. 

Please  indicate  you r  academic  studies 

Answer: 

4. 

How  long  have  you  been  programming? 

Answer: 

Name 


□ass 


(Veer  /  Month) 


Oe-motivatlng 

Frustrating 

Unengaging 

Difficult 


Uninteresting 

De-motivatEng 

Frustrating 

Unengaging 

Difficult 


1  think  the  Lecture/Discussion  sessions  were... 

1  think  the  lab  sessions  were... 

Interesting 

Motivating 

inspiring 

Engaging 

Easy 


Interesting 

Motivating 

inspiring 

Engaging 

Easy 


1  think  that  the  connection  between  Jecture/disnissten  and  lab  was... 

Irrelevant 

Closely  related 

Unhelpful 

Helpful 
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Appendix  C:  Learning  Self-Assessment  Survey  Result 
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Appendix  D:  Pre-  /  Post-  Test  data 


Table  1.  Pre-test  scores  across  participants 
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Table  2.  Post-test  scores  across  participants;  improvement  over  Pre-test 
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